[Stack 13/27] Vectorize participant info computation (3-15x speedup) by jucor · Pull Request #2437 · compdemocracy/polis

jucor · 2026-03-11T13:10:29Z

Summary

Stacked on #2436 (Speed up regression tests). Please review and merge #2436 first.
Next in stack: #2443 (Fix test DB connection: use DATABASE_URL with dotenv)

Replaces the O(N×G×C) per-participant Python loop in _compute_participant_info_optimized with bulk NumPy operations: matrix-wide vote counting (np.sum over axis) and per-group Pearson correlation via P @ g matrix multiply
Adds 31 unit tests covering vote counts, group correlations, edge cases (small groups, zero-std, NaN handling, missing members), and golden snapshot regression
Correlations now return Python float instead of numpy.float64
Includes a benchmark script (scripts/benchmark_participant_info.py) that runs old vs new on the same data

Benchmark results

Measured on real datasets (5 runs, median), old per-participant loop vs new vectorized:

Dataset	Size	Old	New	Speedup
vw	69p × 125c × 4g	0.011s	0.001s	14.6x
biodiversity	536p × 314c × 2g	0.047s	0.006s	8.1x
(larger private datasets)				3–6x

Speedup is higher on smaller datasets (loop overhead dominates) and lower on very large ones (matrix materialization dominates). Overall 3–15x depending on size.

Test plan

31 unit tests pass (pre-vectorization baseline established first, then re-run post)
Golden snapshot regression passes for biodiversity + vw
Full regression test suite passes (40/40)
Benchmark run on all datasets including private (results above)
Max correlation diff across all datasets: < 2e-15

🤖 Generated with Claude Code

Copilot

Pull request overview

This PR optimizes Conversation._compute_participant_info_optimized by replacing the per-participant Python loop with vectorized NumPy computations, and adds tests + a benchmark script to validate and measure the speedup.

Changes:

Vectorizes participant vote counting and per-group correlation computation in _compute_participant_info_optimized.
Adds a comprehensive unit test suite for participant info behavior and regression against golden snapshots.
Adds an A/B benchmark script comparing the old loop implementation vs the new vectorized implementation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
`delphi/polismath/conversation/conversation.py`	Replaces per-participant loop with vectorized vote counting and bulk per-group Pearson correlations.
`delphi/tests/test_participant_info.py`	Adds focused unit tests for vote counts, correlations, edge cases, and golden snapshot regression.
`delphi/scripts/benchmark_participant_info.py`	Adds a script to benchmark old vs new implementations on real datasets and sanity-check equivalence.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

delphi/polismath/conversation/conversation.py

delphi/scripts/benchmark_participant_info.py

delphi/tests/test_participant_info.py

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

delphi/tests/test_old_format_repness.py

+        # Create group clusters (vectorized method requires 'center' key)
        group_clusters = [
-            {'id': 1, 'members': ['p1', 'p2']},
-            {'id': 2, 'members': ['p3', 'p4']}
+            {'id': 1, 'members': ['p1', 'p2'], 'center': [0.0]},
+            {'id': 2, 'members': ['p3', 'p4'], 'center': [0.0]}


delphi/tests/test_repness_smoke.py


-from polismath.pca_kmeans_rep.repness import conv_repness, participant_stats
+from polismath.pca_kmeans_rep.repness import conv_repness
+from polismath.conversation.conversation import Conversation


delphi/tests/test_repness_unit.py

+        # Create group clusters (vectorized method requires 'center' key)
        group_clusters = [
-            {'id': 1, 'members': ['p1', 'p2']},
-            {'id': 2, 'members': ['p3', 'p4']}
+            {'id': 1, 'members': ['p1', 'p2'], 'center': [0.0]},
+            {'id': 2, 'members': ['p3', 'p4'], 'center': [0.0]}


github-actions · 2026-03-30T14:01:29Z

Delphi Coverage Report

File	Stmts	Miss	Cover
init.py	2	0	100%
benchmarks/bench_pca.py	76	76	0%
benchmarks/bench_repness.py	81	81	0%
benchmarks/bench_update_votes.py	38	38	0%
benchmarks/benchmark_utils.py	34	34	0%
components/init.py	1	0	100%
components/config.py	165	133	19%
conversation/init.py	2	0	100%
conversation/conversation.py	1107	320	71%
conversation/manager.py	131	42	68%
database/init.py	1	0	100%
database/dynamodb.py	387	234	40%
database/postgres.py	305	205	33%
pca_kmeans_rep/init.py	5	0	100%
pca_kmeans_rep/clusters.py	257	22	91%
pca_kmeans_rep/corr.py	98	17	83%
pca_kmeans_rep/pca.py	52	16	69%
pca_kmeans_rep/repness.py	297	43	86%
pca_kmeans_rep/stats.py	107	22	79%
regression/init.py	4	0	100%
regression/clojure_comparer.py	188	17	91%
regression/comparer.py	887	720	19%
regression/datasets.py	135	27	80%
regression/recorder.py	36	27	25%
regression/utils.py	138	87	37%
run_math_pipeline.py	260	114	56%
umap_narrative/500_generate_embedding_umap_cluster.py	210	109	48%
umap_narrative/501_calculate_comment_extremity.py	112	54	52%
umap_narrative/502_calculate_priorities.py	135	135	0%
umap_narrative/700_datamapplot_for_layer.py	502	502	0%
umap_narrative/701_static_datamapplot_for_layer.py	310	310	0%
umap_narrative/702_consensus_divisive_datamapplot.py	432	432	0%
umap_narrative/801_narrative_report_batch.py	785	785	0%
umap_narrative/802_process_batch_results.py	265	265	0%
umap_narrative/803_check_batch_status.py	175	175	0%
umap_narrative/llm_factory_constructor/init.py	2	2	0%
umap_narrative/llm_factory_constructor/model_provider.py	157	157	0%
umap_narrative/polismath_commentgraph/init.py	1	0	100%
umap_narrative/polismath_commentgraph/cli.py	270	270	0%
umap_narrative/polismath_commentgraph/core/init.py	3	3	0%
umap_narrative/polismath_commentgraph/core/clustering.py	108	108	0%
umap_narrative/polismath_commentgraph/core/embedding.py	104	104	0%
umap_narrative/polismath_commentgraph/lambda_handler.py	219	219	0%
umap_narrative/polismath_commentgraph/schemas/init.py	2	0	100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py	160	9	94%
umap_narrative/polismath_commentgraph/tests/conftest.py	17	17	0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py	74	74	0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py	55	55	0%
umap_narrative/polismath_commentgraph/tests/test_storage.py	87	87	0%
umap_narrative/polismath_commentgraph/utils/init.py	3	0	100%
umap_narrative/polismath_commentgraph/utils/converter.py	283	237	16%
umap_narrative/polismath_commentgraph/utils/group_data.py	354	336	5%
umap_narrative/polismath_commentgraph/utils/storage.py	584	477	18%
umap_narrative/reset_conversation.py	159	50	69%
umap_narrative/run_pipeline.py	453	312	31%
utils/general.py	62	41	34%
Total	10877	7600	30%

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-30T16:31:15Z

delphi/tests/test_participant_info.py

+        g = golden_pi[pid_str]
+        # Find matching pid in computed (could be int or str)
+        c = computed_pi.get(int(pid_str), computed_pi.get(pid_str))
+        assert c is not None, f"Missing participant {pid_str} in computed"


In test_participant_info_matches_golden, int(pid_str) will raise ValueError if participant IDs in the golden snapshot are non-numeric (possible for some datasets / local fixtures). Consider wrapping the int(...) cast in a try/except (or checking .isdigit()) and falling back to the string key to keep the test robust to non-integer IDs.

Copilot · 2026-03-30T16:31:16Z

delphi/tests/test_participant_info.py

+    votes_dict, metadata = prepare_votes_data(dataset_name)
+
+    # Run full pipeline
+    conv = Conversation(dataset_name)
+    conv = conv.update_votes(votes_dict, recompute=True)
+


This golden snapshot test runs a full Conversation.update_votes(..., recompute=True) pipeline per discovered dataset, which appears redundant with tests/test_regression.py::test_conversation_regression (it already computes after_full_recompute via to_dict() which includes participant_info). Consider reusing the regression comparer output / golden stage data, or marking this as a slow/optional test, to avoid adding another full pipeline run per dataset.

Copilot · 2026-03-30T16:31:16Z

delphi/tests/test_repness_unit.py

-        
-        # Create group clusters
+
+        # Create group clusters (vectorized method requires 'center' key)


The comment says the vectorized method requires a 'center' key, but _compute_participant_info_optimized() only reads id and members. Consider adjusting the comment to avoid implying 'center' is required (or explain that it's just matching the usual group_clusters schema).

Suggested change

# Create group clusters (vectorized method requires 'center' key)

# Create group clusters (include 'center' to match usual group_clusters schema)

Copilot · 2026-03-30T16:31:16Z

delphi/tests/test_old_format_repness.py

        vote_matrix = pd.DataFrame(vote_data, index=row_names, columns=col_names)

-        # Create group clusters
+        # Create group clusters (vectorized method requires 'center' key)


The comment says the vectorized method requires a 'center' key, but _compute_participant_info_optimized() does not reference 'center' (it only uses id and members). Consider updating the comment to avoid suggesting 'center' is required, or clarify it's only to match the typical group_clusters schema.

Suggested change

# Create group clusters (vectorized method requires 'center' key)

# Create group clusters (including optional 'center' key to match typical schema)

… imports - Clarify Beta(2,2) comment to specify posterior mode (MAP) estimate - Import PSEUDO_COUNT from repness.py instead of hardcoding in simplified_repness_test.py - Move PSEUDO_COUNT import to module-level in test_repness_unit.py and test_old_format_repness.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Map plan's "PR N" labels to actual GitHub PR numbers and stack positions. Add non-discrepancy PRs table. Replace old naming convention with stack-managed [Stack N/M] prefix. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the O(N×G×C) per-participant Python loop with bulk NumPy operations: matrix-wide vote counting and per-group Pearson correlation via matrix multiply. The assembly loop only indexes pre-computed arrays. Also adds 31 unit tests (test_participant_info.py) covering vote counts, group correlations, edge cases, and golden snapshot regression. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

A/B benchmark comparing old per-participant loop vs new vectorized implementation. Measured 3-15x speedup depending on dataset size. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ype assert - Remove unused group_vote_matrices dict (comment 3) - Validate --runs >= 1 in benchmark script (comment 4) - Tighten correlation type check to exact `float` (comment 5) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Delete the O(participants × groups × members) participant_stats() from repness.py (~130 lines) now that _compute_participant_info_optimized() on the Conversation class provides a 3-15x faster NumPy replacement. - Remove function from repness.py and __init__.py exports - Update all imports (conversation.py, run_analysis.py, 4 test files) - Rewrite 3 test methods to call the vectorized Conversation method - Add "remove dead code after replacement" principle to the plan Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jucor · 2026-03-30T22:54:36Z

Superseded by spr-managed PR stack. See the new stack starting at #2508.

jucor requested review from ballPointPenguin and whilo March 11, 2026 13:10

jucor changed the title ~~Vectorize participant info computation (~100x speedup)~~ [Stack 11/11] Vectorize participant info computation (~100x speedup) Mar 11, 2026

jucor changed the title ~~[Stack 11/11] Vectorize participant info computation (~100x speedup)~~ [Stack 11/11] Vectorize participant info computation (3-15x speedup) Mar 11, 2026

jucor requested a review from Copilot March 11, 2026 13:22

Copilot started reviewing on behalf of jucor March 11, 2026 13:22 View session

Copilot AI reviewed Mar 11, 2026

View reviewed changes

jucor requested a review from Copilot March 13, 2026 12:56

Copilot started reviewing on behalf of jucor March 13, 2026 12:57 View session

Copilot AI reviewed Mar 13, 2026

View reviewed changes

jucor force-pushed the jc/regression-test-perf branch from 013624a to 19d4a5b Compare March 13, 2026 13:09

jucor force-pushed the jc/vectorize-participant-info branch from b839591 to 3048f93 Compare March 13, 2026 13:09

jucor changed the title ~~[Stack 11/11] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/12] Vectorize participant info computation (3-15x speedup) Mar 13, 2026

This was referenced Mar 13, 2026

[Stack 12/27] Speed up regression tests #2436

Closed

[Stack 14/27] Fix test DB connection: use DATABASE_URL with dotenv #2443

Closed

jucor force-pushed the jc/regression-test-perf branch from 19d4a5b to 4a57515 Compare March 13, 2026 13:50

jucor force-pushed the jc/vectorize-participant-info branch from 3048f93 to e883338 Compare March 13, 2026 13:53

jucor changed the title ~~[Stack 11/12] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/13] Vectorize participant info computation (3-15x speedup) Mar 13, 2026

jucor force-pushed the jc/regression-test-perf branch from 4a57515 to 967ffec Compare March 13, 2026 14:13

jucor force-pushed the jc/vectorize-participant-info branch from e883338 to 9dee5d9 Compare March 13, 2026 14:15

jucor force-pushed the jc/regression-test-perf branch from 967ffec to 4b803ca Compare March 13, 2026 15:56

jucor force-pushed the jc/vectorize-participant-info branch from 9dee5d9 to 0d4cd71 Compare March 13, 2026 15:56

jucor changed the title ~~[Stack 11/13] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/15] Vectorize participant info computation (3-15x speedup) Mar 16, 2026

jucor force-pushed the jc/regression-test-perf branch from 4b803ca to f4252a0 Compare March 16, 2026 16:04

jucor force-pushed the jc/vectorize-participant-info branch from 0d4cd71 to fa345cd Compare March 16, 2026 16:04

jucor changed the title ~~[Stack 11/15] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/16] Vectorize participant info computation (3-15x speedup) Mar 16, 2026

jucor changed the title ~~[Stack 11/16] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/17] Vectorize participant info computation (3-15x speedup) Mar 16, 2026

jucor changed the title ~~[Stack 11/17] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/24] Vectorize participant info computation (3-15x speedup) Mar 17, 2026

jucor changed the title ~~[Stack 11/24] Vectorize participant info computation (3-15x speedup)~~ [Stack 11/25] Vectorize participant info computation (3-15x speedup) Mar 17, 2026

jucor force-pushed the jc/regression-test-perf branch from f4252a0 to 3f09ab0 Compare March 19, 2026 10:43

jucor force-pushed the jc/regression-test-perf branch from 478fa73 to ce0c874 Compare March 27, 2026 01:15

jucor force-pushed the jc/vectorize-participant-info branch from 923d07c to 5c9399b Compare March 27, 2026 01:15

jucor force-pushed the jc/regression-test-perf branch from ce0c874 to da4b4de Compare March 27, 2026 02:10

jucor force-pushed the jc/vectorize-participant-info branch from 5c9399b to 7c6b83d Compare March 27, 2026 02:10

jucor force-pushed the jc/regression-test-perf branch from da4b4de to a9b000e Compare March 27, 2026 10:41

jucor force-pushed the jc/vectorize-participant-info branch from 7c6b83d to 50a17b4 Compare March 27, 2026 10:41

jucor changed the title ~~[Stack 11/25] Vectorize participant info computation (3-15x speedup)~~ [Stack 12/26] Vectorize participant info computation (3-15x speedup) Mar 30, 2026

jucor force-pushed the jc/regression-test-perf branch from a9b000e to 41dec2d Compare March 30, 2026 12:48

jucor force-pushed the jc/vectorize-participant-info branch from 50a17b4 to 2ab1789 Compare March 30, 2026 12:48

jucor changed the title ~~[Stack 12/26] Vectorize participant info computation (3-15x speedup)~~ [Stack 13/27] Vectorize participant info computation (3-15x speedup) Mar 30, 2026

jucor force-pushed the jc/regression-test-perf branch from 41dec2d to b3d7506 Compare March 30, 2026 12:54

jucor force-pushed the jc/vectorize-participant-info branch from 2ab1789 to 67731ff Compare March 30, 2026 12:54

jucor requested a review from Copilot March 30, 2026 16:25

Copilot started reviewing on behalf of jucor March 30, 2026 16:26 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

jucor force-pushed the jc/vectorize-participant-info branch from 67731ff to afbd68a Compare March 30, 2026 16:49

jucor force-pushed the jc/regression-test-perf branch from b3d7506 to 5611792 Compare March 30, 2026 16:49

jucor and others added 6 commits March 30, 2026 17:58

Add benchmark script for participant info vectorization

dc37e2b

A/B benchmark comparing old per-participant loop vs new vectorized implementation. Measured 3-15x speedup depending on dataset size. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

jucor force-pushed the jc/regression-test-perf branch from 5611792 to 6a96d48 Compare March 30, 2026 17:05

jucor force-pushed the jc/vectorize-participant-info branch from afbd68a to 6138219 Compare March 30, 2026 17:05

This was referenced Mar 30, 2026

IGNORE -- crash from spr #2497

Closed

IGNORE -- crash from spr #2499

Closed

jucor closed this Mar 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Stack 13/27] Vectorize participant info computation (3-15x speedup)#2437

[Stack 13/27] Vectorize participant info computation (3-15x speedup)#2437
jucor wants to merge 6 commits intojc/regression-test-perffrom
jc/vectorize-participant-info

jucor commented Mar 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions bot commented Mar 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

Copilot AI Mar 30, 2026

Uh oh!

jucor commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		# Create group clusters

		# Create group clusters (vectorized method requires 'center' key)

	# Create group clusters (vectorized method requires 'center' key)
	# Create group clusters (include 'center' to match usual group_clusters schema)

	# Create group clusters (vectorized method requires 'center' key)
	# Create group clusters (including optional 'center' key to match typical schema)

Conversation

jucor commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Benchmark results

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions bot commented Mar 30, 2026

Delphi Coverage Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

jucor commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jucor commented Mar 11, 2026 •

edited

Loading